miracast uibc详解 | 您所在的位置:网站首页 › miracast 接收器 › miracast uibc详解 |
User Input Back Channel 用户输入反向控制通道(UIBC)是一个可选的WFD特性,实现该扩展功能,有助于用户从WFD sink端控制WFD source端的通信。 wifi display spec文档定义: 4.11 User Input Back Channel The User Input Back Channel (UIBC) is an optional WFD feature that when implemented facilitates communication of user inputs to a User Interface, present at the WFD Sink, to the WFD Source. All UIBC user inputs are packetized using a common packet header and transported over TCP/IP. The user input categories include Generic, and HIDC. The Generic category is used for device agnostic user inputs that are processed at the application level. Generic user inputs are formatted using the Generic Input Body. HIDC is used for user inputs generated by HIDs like remote control, keyboard, etc.[46]. HIDC user inputs are formatted using the HIDC Input Body. 翻译一下 4.11 UIBC UIBC是WFD的可选feature,便于用户在UI上用户交流?展现在sink端,到source端?什么鬼! 所有uibc用户输入被打包成common的包头,然后用tcp/ip进行传输,user input类别包括HIDC和Generic。 Generic类别被用于在应用程序级别处理的与设备无关的用户输入。 HIDC被用于通过HIDs(一类HID类设备)生成的user input,像远程控制,键盘等等,UIDC user inputs被用UIDC Input Body来格式化 虽然翻译的乱七八糟,但还是大概有了点概念。 接下来是 1.1 UIBC的TCP/IP封包形式:
UIBC是使用RTSP GET_PARAMETER 和SET_PARAMETER消息建立和维护的。 消息序列如下两个Figure
github中 hw手机fw代码uibc部分研究 代码如下:反编译出来的代码 https://github.com/SivanLiu/HwFrameWorkSource/blob/5b92ed0f1ccb4bafc0fdb08b6fc4d98447b754ad/Mate20_9_0_0/src/main/java/com/android/server/display/HwUibcReceiver.java(疑似华为手机source uibc的实现) 3.1 创建UIBC接收器WifiDisplayController.this.mUibcInterface.createReceiver,调用的是HwUibcReceiver中的createReceiver方法
遍历所有的Event,然后分别调用InputManager的injectInputEvent接口,注入事件 代码地址: https://github.com/albfan/miraclecast/tree/master/src/uibc 看上去只是一个控制台的二进制实现,并不是电视端的,但是大致原理应该是相通的
前面组完包之后就调用sendUibcMessage进行发送,第二个参数为4.1中创建的socket连接fd。 int sendUibcMessage(UibcMessage* uibcmessage, int sockfd) { ssize_t n; printf("sending %zu bytes\n", uibcmessage->m_PacketDataLen); n = write(sockfd, uibcmessage->m_PacketData , uibcmessage->m_PacketDataLen); if (n if (FeatureOption.MTK_WFD_SINK_UIBC_SUPPORT) { mGlobal.sendUibcInputEvent(input); } } 5.2 frameworks/av /av/media/libmediaplayerservice/RemoteDisplay.cpp status_t RemoteDisplay::sendUibcEvent(const String8& eventDesc) { const char* pEventDes = eventDesc.string(); // 取一个字节。直接转换 int type = *pEventDes; if (*(pEventDes+1) != ',') return BAD_VALUE; ALOGD("sendUibcEvent: type=0x%X", type); switch (type) { //0x30是数字0字符的ascii码 case 0x30: case 0x31: case 0x32: mSink->sendUIBCGenericTouchEvent(pEventDes); break; //0x33是数字3字符的ascii码 case 0x33: case 0x34: mSink->sendUIBCGenericKeyEvent(pEventDes); break; default: return BAD_VALUE; } return OK; } 5.4 frameworks-ext/av status_t WifiDisplaySink::sendUIBCGenericTouchEvent(const char * eventDesc) { status_t err; if (mUibcClientHandler == NULL) return -1; err = mUibcClientHandler->sendUibcMessage(mNetSession, UibcMessage::GENERIC_TOUCH_DOWN, eventDesc); return err; }到这里就很全面了,过程推测应该和前面4中控制台的基本一样。毕竟函数名字都一模一样。也不知道谁抄的谁。 还有个遗留问题,应用是怎么调用的sendUibcInputEvent接口。找到的结果如下 5.5 packages/apps/Settings // /src/com/mediatek/settings/wfd/WfdSinkSurfaceFragment.java @Override public boolean onTouchEvent(MotionEvent ev) { ... StringBuilder eventDesc = new StringBuilder(); eventDesc.append( //如果是uo的时候,这里就用 //GENERIC_INPUT_TYPE_ID_TOUCH_UP //这里就不用ascii码了 //因为0转换完之后就是0x30了 String.valueOf(GENERIC_INPUT_TYPE_ID_TOUCH_DOWN)) .append(","); eventDesc.append(getTouchEventDesc(ev)); sendUibcInputEvent(eventDesc.toString()); ... } @Override public boolean onKeyDown(int keyCode, KeyEvent event) { ... int asciiCode = event.getUnicodeChar(); if (asciiCode == 0 || asciiCode WfdRtspUibcPacket* newPacket = new WfdRtspUibcPacket(); currEventType = WfdRtspProtocol::instance().mUibcEventType; newPacket->setCategory(WfdRtspUibcPacket::UIBC_HIDC); if (WfdRtspProtocol::instance().mUibcEventType == HIDC_EVT_TYPE_MOUSE) newPacket->setHidType(WfdRtspUibcPacket::HID_TYPE_MOUSE); else newPacket->setHidType(WfdRtspUibcPacket::HID_TYPE_KEYBOARD); UibcQueue.push_back(newPacket); ... WfdRtspUibcPacket* uibcPacket = UibcQueue.front(); UibcQueue.pop_front(); if (WfdRtspProtocol::instance().mUibcEventType == HIDC_EVT_TYPE_MOUSE) { /* -------------------------------------------------------------------- HID Mouse/USB -------------------------------------------------------------------*/ // displacement是移位的意思 int disps[2] = {5, 5}; /* x-displacement, y-displacement */ // 循环30次 for (round_count = 0; round_count WFD_LOG_ERR("Error! send UIBC Mouse data1 error !"); break; } total_counts ++; disps[0] += 5; disps[1] += 5; // sleep 500ms usleep(DELAY_BETWEEN_EACH_EVENT_MS * 1000); ... } } }这段代码主要是循环发送了一个鼠标事件,坐标值从5,5一直自增加5,增加三十次,看上去是轨迹是一个斜率为1的一条斜线。 继续看下这个关键方法getTranData,这个方法完成的就是组包操作 int WfdRtspUibcPacket::getTranData(unsigned char * buffer,int len, void *data) { ... else if (mCategory == UIBC_HIDC) { struct uibc_packet_s upkt; unsigned short *usptr; unsigned char *buf_ptr = NULL, *buf_head_ptr = NULL; memset(buffer, 0, len); memset(&upkt, 0, sizeof(struct uibc_packet_s)); // 构造包头 upkt.hdr.version = mVersion; upkt.hdr.T = 0; upkt.hdr.rsvd = 0; upkt.hdr.input_category = mCategory; // 转换网络顺序 /* need to convert first ushort bytes to network order */ usptr = (unsigned short *)&upkt.hdr; *usptr = htons(*usptr); if (mHidType == HID_TYPE_MOUSE) { struct hid_mouse_input_report_s *mouse = NULL; //坐标指针 int *disps = (int *)data; // 必须走USB? // 其他选项有INFRARED红外线 // BT WIFI等等 // 我理解这里是采用数据流模拟USB输入的意思 upkt.u_body.uibc_hidc.input_path = HID_INPUT_PATH_USB; upkt.u_body.uibc_hidc.hid_type = HID_TYPE_MOUSE; upkt.u_body.uibc_hidc.usage = 0; /* not Report Descriptor */ upkt.u_body.uibc_hidc.length = htons(HIDC_USB_MOUSE_VALUE_LEN); // 构造鼠标hid report descriptor // 从这里来看,mouse直接指向了hidc_value的首地址 mouse = (struct hid_mouse_input_report_s *)&upkt.u_body.uibc_hidc.hidc_value[0]; mouse->x_disp = (unsigned char)disps[0]; mouse->y_disp = (unsigned char)disps[1]; // 这里提示要注意字节对齐的问题 /* copy result to buffer */ /* Note that because alignment issue that after HIDC "usage filed" will have one additional byte, so we manage the buffer by copying. It is better to use pragma though. */ buf_ptr = buffer; buf_head_ptr = buf_ptr; // 跳过uibc包头 buf_ptr += sizeof(struct uibc_packet_hdr_s); *buf_ptr++ = upkt.u_body.uibc_hidc.input_path; *buf_ptr++ = upkt.u_body.uibc_hidc.hid_type; *buf_ptr++ = upkt.u_body.uibc_hidc.usage; memcpy(buf_ptr, (unsigned char *)&upkt.u_body.uibc_hidc.length, sizeof(upkt.u_body.uibc_hidc.length)); // 跳到hidc buf_ptr += sizeof(upkt.u_body.uibc_hidc.length); // 填充hid report descriptor memcpy(buf_ptr, mouse, sizeof(struct hid_mouse_input_report_s)); buf_ptr += sizeof(struct hid_mouse_input_report_s); // 4+9 retLen = UIBC_HDR_LEN + UIBC_HIDC_BODY_LEN_MOUSE; // 主机字节序变为网络字节序 // 网络字节顺序采用big-endian排序方式 upkt.hdr.length = htons((unsigned short)retLen); memcpy(buf_head_ptr, (unsigned char *)&upkt.hdr, sizeof(struct uibc_packet_hdr_s)); } } }上面这些复杂的结构体看名字头都晕。可以看下定义 struct uibc_packet_s { struct uibc_packet_hdr_s hdr; // 这里是二选一的意思 union { struct uibc_body_generic_s uibc_gen; struct uibc_body_hidc_s uibc_hidc; } u_body; }; struct uibc_packet_hdr_s { // 冒号这是c语言的写法,位域 // 表示占用4位 unsigned short input_category : 4; /* Input Category: b12~15 */ unsigned short rsvd : 8; /* Reserved: b4~11 */ unsigned short T : 1; /* T: b3 */ unsigned short version : 3; /* version: b0~2 */ unsigned short length; /* Length: b16~31 */ /* unsigned short timestamp; */ /* Timestamp(Optional): b32~47*/ }; // 这个是generic的 struct uibc_body_generic_s { unsigned char input_type_id; unsigned short length; unsigned char describe[64]; }; // 这一部分才是属于hidc的 struct uibc_body_hidc_s { unsigned char input_path; unsigned char hid_type; unsigned char usage; unsigned short length; // 如果是描述符这种形式,hid_mouse_input_report_s是直接放在这里的 unsigned char hidc_value[64]; }; struct hid_mouse_input_report_s { unsigned char buttons; unsigned char x_disp; unsigned char y_disp; unsigned char dev_specific; };到这里,理论上来说功能已经可以实现了。按照固定格式组好13个字节(头一共32位,四个字节。body9个字节uibc_body_hidc_s由1+1+1+2+4(report desc)组成)的包,然后通过socket发送出去应该就好了。 7.2 实际开发中的坑实际开发下来,发现hid_mouse_input_report_s这个结构体,后面dev_specifics字段应该拿掉。参考: 搜索之后发现确实有这个说法。也就是说前面一个字节的button其实八个byte都是有意义的。又去翻了下Wi-Fi_Display_Technical_Specification_v2.1_0,又发现点新东西 一些相关博客 https://blog.csdn.net/sinat_37343534/article/details/116306013 https://m.xuejianbihua.com/item/xYTE3ZDUyODVhNzJjMTY4MDk4YzhmMmY2u.html https://blog.csdn.net/sinat_37343534/article/details/116446980 控制台版本的sink端代码实现 https://github.com/albfan/miraclecast/tree/master/src/uibc 疑似华为手机source uibc的实现代码 https://github.com/SivanLiu/HwFrameWorkSource/blob/5b92ed0f1ccb4bafc0fdb08b6fc4d98447b754ad/Mate20_9_0_0/src/main/java/com/android/server/display/HwUibcReceiver.java lg电视的实现效果 https://www.youtube.com/watch?v=Q-I9g9g_21g 苏州必捷的实现效果 https://zhuanlan.zhihu.com/p/437928388 这下面有client代码 ~/code/giant-mtk/vendor/mediatek/proprietary_tv/open/hardware/wfd_client https://www.usb.org/hid usb的hid鼠标键盘报告描述符: http://t.zoukankan.com/zongzi10010-p-10155333.html https://www.usb.org/sites/default/files/hid1_11.pdf Android鼠标源码研究(五)–输入事件处理 https://blog.51cto.com/u_15067266/2908866 Tutorial about USB HID Report Descriptors https://eleccelerator.com/tutorial-about-usb-hid-report-descriptors/ HID鼠标描述符 https://www.luliang.vip/archives/4.html |
CopyRight 2018-2019 实验室设备网 版权所有 |